Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference

Large language models (LLMs) based on transformers have made significant strides in recent years, the success of which is driven by scaling up their model size. Despite their high algorithmic performance, the computational and memory requirements of LLMs present unprecedented challenges. To tackle the high compute requirements of LLMs, the Mixture-of-Experts (MoE) architecture was introduced which is able to scale its model size without proportionally scaling up its computational requirements. Unfortunately, MoE's high memory demands and dynamic activation of sparse experts restrict its applicability to real-world problems. Previous solutions that offload MoE's memory-hungry expert parameters to CPU memory fall short because the latency to migrate activated experts from CPU to GPU incurs high performance overhead. Our proposed Pre-gated MoE system effectively tackles the compute and memory challenges of conventional MoE architectures using our algorithm-system co-design. Pre-gated MoE employs our novel pre-gating function which alleviates the dynamic nature of sparse expert activation, allowing our proposed system to address the large memory footprint of MoEs while also achieving high performance. We demonstrate that Pre-gated MoE is able to improve performance, reduce GPU memory consumption, while also maintaining the same level of model quality. These features allow our Pre-gated MoE system to cost-effectively deploy large-scale LLMs using just a single GPU with high performance

Performance analysis of a polling model with BMAP and across-queue state-dependent service discipline

As various video services become popular, video streaming will dominate the mobile data traffic. The H.264 standard has been widely used for video compression. As the successor to H.264, H.265 can compress video streaming better, hence it is gradually gaining market share. However, in the short term H.264 will not be completely replaced, and will co-exist with H.265. Using H.264 and H.265 standards, three types of frames are generated, and among different types of frames exist dependencies. Since the radio resources are limited, using dependencies and quantities of frames in buffers, an appropriate time division transmission policy can be applied to transmit different types of frames sequentially, in order to avoid the occurrence of video carton or decoding failure. Polling models with batch Markovian arrival process (BMAP) and across-queue state-dependent service discipline are considered to be effective means in the design and optimization of appropriate time division transmission policies. However, the BMAP and across-queue state-dependent service discipline of the polling models lead to the large state space and several coupled state transition processes, which complicate the performance analysis. There have been very few researches in this regard. In this paper, a polling model of this type is analyzed. By constructing a supplementary embedded Markov chain and applying the matrix-analytic method based on the semi-regenerative process, the expressions of important performance measures including the joint queue length distribution, the customer blocking probability and the customer mean waiting time are obtained. The analysis will provide inspiration for analyzing the polling models with BMAP and across-queue state-dependent service discipline, to guide the design and optimization of time division transmission policies for transmitting the video compressed by H.264 and H.265

Ab Initio Studies on Interactions in K3_3C60_{60} under High Pressure

Fullerene solids doped with alkali metals (A$_3$C$_{60}$, A = K, Rb, Cs) exhibit a superconducting transition temperature ($T_c$) as high as 40 K, and their unconventional superconducting properties have been a subject of debate. With application of high pressure on K$_3$C$_{60}$ and Rb$_3$C$_{60}$, the experiments demonstrate the decrease of $T_c$. In this paper, we focus on K$_3$C$_{60}$ and derive the structure of K$_3$C$_{60}$ under different pressures based on first-principles calculations, exploring the trends of Coulomb interactions at various pressures. By utilizing the Maximally Localized Wannier function approach, Constrained Density Functional Perturbation Theory (cDFPT), and Constrained Random Phase Approximation (cRPA), we construct a microscopic low-energy model near the Fermi level. Our results strongly indicate that, in the K$_3$C$_{60}$ system, as pressure increases, the effect of phonons is the key to intraorbital electron pairing. There is a dominance of the phonon-driven superconducting mechanism at high pressure

Non-Markovian Transmission through Two Quantum Dots Connected by a Continuum

We consider a transport setup containing a double-dot connected by a continuum. Via an exact solution of the time-dependent Schr\"odinger equation, we demonstrate a highly non-Markovian quantum-coherence-mediated transport through this dot-continuum-dot (DCD) system, which is in contrast with the common premise since in typical case a quantum particle does not reenter the system of interest once it irreversibly decayed into a continuum (such as the spontaneous emission of a photon). We also find that this DCD system supports an unusual steady state with unequal source and drain currents, owing to electrons irreversibly entering the continuum and floating there

CagA-positive Helicobacter pylori may promote and aggravate scrub typhus

Helicobacter pylori (H. pylori) infection may alter the host’s resistance to tsutsugamushi disease pathogens through the Th1 immune response, leading to potential synergistic pathogenic effects. A total of 117 scrub typhus cases at Beihai People’s Hospital and affiliated hospitals of Youjiang University for Nationalities and Medical Sciences were studied from January to December 2022, alongside 130 healthy individuals forming the control group. All participants underwent serum H. pylori antibody testing. The prevalence of H. pylori infection was significantly higher among scrub typhus patients (89.7%) compared to healthy individuals (54.6%) (p < 0.05). Moreover, type I H. pylori infection was notably more prevalent in scrub typhus cases (67.5%) compared to healthy individuals (30%) (p < 0.05). Multifactorial analysis demonstrated type I H. pylori infection as an independent risk factor for scrub typhus (adjusted odds ratio: 2.407, 95% confidence interval: 1.249–4.64, p = 0.009). Among scrub typhus patients with multiple organ damage, the prevalence of type I H. pylori infection was significantly higher (50.6%) than type II H. pylori infection (15.4%) (χ2 = 4.735, p = 0.030). These results highlight a higher incidence of H. pylori infection in scrub typhus patients compared to the healthy population. Additionally, type I H. pylori strain emerged as an independent risk factor for scrub typhus development. Moreover, individuals infected with type I H. pylori are more susceptible to multiple organ damage. These findings suggest a potential role of H. pylori carrying the CagA gene in promoting and exacerbating scrub typhus

A juvenile mouse model of anti-N-methyl-D-aspartate receptor encephalitis by active immunization

IntroductionAnti-N-methyl-D-aspartate receptor (NMDAR) encephalitis is a common autoimmune encephalitis, and it is associated with psychosis, dyskinesia, and seizures. Anti-NMDAR encephalitis (NMDARE) in juveniles and adults presents different clinical charactreistics. However, the pathogenesis of juvenile anti-NMDAR encephalitis remains unclear, partly because of a lack of suitable animal models.MethodsWe developed a model of juvenile anti-NMDAR encephalitis using active immunization with an amino terminal domain peptide from the GluN1 subunit (GluN1356 − 385) against NMDARs in 3-week-old female C57BL/6J mice.ResultsImmunofluorescence staining suggested that autoantibody levels in the hippocampus increased, and HEK-293T cells staining identified the target of the autoantibodies as GluN1, suggesting that GluN1-specific immunoglobulin G was successfully induced. Behavior assessment showed that the mice suffered significant cognition impairment and sociability reduction, which is similar to what is observed in patients affected by anti-NMDAR encephalitis. The mice also exhibited impaired long-term potentiation in hippocampal CA1. Pilocarpine-induced epilepsy was more severe and had a longer duration, while no spontaneous seizures were observed.ConclusionThe juvenile mouse model for anti-NMDAR encephalitis is of great importance to investigate the pathological mechanism and therapeutic strategies for the disease, and could accelerate the study of autoimmune encephalitis

Search for the decay J/ψ→γ+invisibleJ/\psi\to\gamma + \rm {invisible}

We search for $J/\psi$ radiative decays into a weakly interacting neutral particle, namely an invisible particle, using the $J/\psi$ produced through the process $\psi(3686)\to\pi^+\pi^-J/\psi$ in a data sample of $(448.1\pm2.9)\times 10^6$ $\psi(3686)$ decays collected by the BESIII detector at BEPCII. No significant signal is observed. Using a modified frequentist method, upper limits on the branching fractions are set under different assumptions of invisible particle masses up to 1.2 $\mathrm{\ Ge\kern -0.1em V}/c^2$. The upper limit corresponding to an invisible particle with zero mass is 7.0$\times 10^{-7}$ at the 90\% confidence level

Search for the reaction e+e−→χcJπ+π− and a charmoniumlike structure decaying to χcJπ± between 4.18 and 4.60 GeV

We search for the process e+e-→χcJπ+π- (J=0, 1, 2) and for a charged charmoniumlike state in the χcJπ± subsystem. The search uses datasets collected with the BESIII detector at the BEPCII storage ring at center-of-mass energies between 4.18 GeV and 4.60 GeV. No significant χcJπ+π- signals are observed at any center-of-mass energy, and thus upper limits are provided which also serve as limits for a possible charmoniumlike structure in the invariant χcJπ± mass